Imaging apparatus and image sensor array

ABSTRACT

An imaging apparatus including an imaging lens, and an image sensor array of first and second image sensor units, wherein a single first image sensor unit includes a single first microlens and a plurality of image sensors, a single second image sensor unit includes a single second microlens and a single image sensor, light passing through the imaging lens and reaching each first image sensor unit passes through the first microlens and forms an image on the image sensors constituting the first image sensor unit, light passing through the imaging lens and reaching each second image sensor unit passes through the second microlens and forms an image on the image sensor constituting the second image sensor unit, an inter-unit light shielding layer is formed between the image sensor units, and a light shielding layer is not formed between the image sensor units constituting the first image sensor unit.

RELATED APPLICATION DATA

This application is a continuation of U.S. patent application Ser. No.16/565,609 filed Sep. 10, 2019, which is a continuation of U.S. patentapplication Ser. No. 16/161,332 filed Oct. 16, 2018, now U.S. Pat. No.10,425,632 issued Sep. 24, 2019, which is a continuation of U.S. patentapplication Ser. No. 15/015,427 filed Feb. 4, 2016 now U.S. Pat. No.10,129,531 issued Nov. 13, 2018, which is a continuation of U.S. patentapplication Ser. No. 13/771,865, filed Feb. 20, 2013, now U.S. Pat. No.9,294,756 issued Mar. 22, 2016, the entireties of which are incorporatedherein by reference to the extent permitted by law. The presentapplication claims the benefit of priority to Japanese PatentApplication No. JP 2012-034974, filed on Feb. 21, 2012 in the JapanPatent Office, the entirety of which is incorporated by reference hereinto the extent permitted by law.

BACKGROUND

The present disclosure relates to an imaging apparatus and an imagesensor array, and more particularly, to an imaging apparatus that imagesa subject as a stereo image, and to an image sensor array used in suchan imaging apparatus.

For some time, various imaging apparatus have been proposed anddeveloped. Additionally, there have been proposed imaging apparatusconfigured to perform given image processing on an imaging signalobtained by imaging and output the result, such as an imaging apparatusthat images a subject as a stereo image, for example. For example,International Publication No. 06/039486 proposes an imaging apparatusthat uses a technique called light field photography. The imagingapparatus includes an imaging lens, a microlens array (light fieldlens), a photosensitive element, and an image processor. An imagingsignal obtained from the photosensitive element includes information onthe light intensity at the photosensitive surface of the photosensitiveelement, as well as information on the travel direction of that light.Additionally, on the basis of such an imaging signal, an observed imagefrom an arbitrary viewpoint or direction may be reconstructed in theimage processor to obtain a stereo image.

Alternatively, there has been proposed a system in which a commonsubject is simultaneously imaged by two video cameras disposed to theleft and right of each other, and a stereo image is displayed bysimultaneously outputting the two images thus obtained (a right-eyeimage and a left-eye image). However, when two video cameras are used inthis way, the apparatus becomes bulky and is impractical. Moreover, thebaseline between the two video cameras, or in other words, theinterpupillary distance of the stereo camera, is often set toapproximately 65 mm, equivalent to the distance between the human eyes,irrespective of the lens zoom ratio. In such cases, binocular parallaxincreases for close-up images and forces the viewer's visual system toprocess information differently from daily life, which can lead toeyestrain. Additionally, imaging a moving subject with two video camerasinvolves precise synchronization control of the two video cameras and isextremely difficult. Accurate control of the angle of convergence isalso extremely difficult.

In order to make adjusting the lens system easier for stereophotography, there has been proposed a stereo photography apparatus thatuses a shared optical system by introducing a polarization filter whichpolarizes incoming light such that respective rays become orthogonal toeach other (see Japanese Examined Patent Application Publication No.H6-054991, for example).

Also proposed is a technique that conducts stereo photography with animaging apparatus made up of two lenses and one imaging means (seeJapanese Unexamined Patent Application Publication No. 2004-309868, forexample). The imaging apparatus disclosed in this patent applicationpublication is provided with

imaging means in which a number of pixels corresponding to an integermultiple of a given number of scanlines are provided on an imagingsurface,

first horizontal component polarizing means that transmits onlyhorizontal components in first image light from a subject, and

first vertical component polarizing means, disposed separated from thefirst horizontal component polarizing means by a given distance, thattransmits only vertical components in second image light from thesubject,

wherein the horizontal components transmitted by the first horizontalcomponent polarizing means are condensed onto a given range of pixels onthe imaging surface, and

the vertical components transmitted by the first vertical componentpolarizing means are condensed onto a remaining range of pixels thatexcludes the given range. Specifically, a horizontal componentpolarizing filter and a vertical component polarizing filter aredisposed separated by an interval corresponding to the parallax of thehuman eyes, and provided together with two lenses at a positionseparated from the imaging surface of a CCD by a given distance.

SUMMARY

However, the imaging apparatus disclosed in International PublicationNo. 06/039486 is problematic in that the imaging apparatus is bulky dueto an increase in the number of components. Additionally, with therecent advances in higher definition (i.e., smaller pixels) for imagingapparatus, light field lenses demand extremely precise positioning.

Meanwhile, with the technology disclosed in Japanese Examined PatentApplication Publication No. H6-054991, the lens system is shared byoverlapping the output from two polarizing filters to merge the opticalpaths. However, additional polarizing filters are provided downstreamfor extraction of a right-eye image and a left-eye image, and theoptical path itself is again split in order for the light to enter theseparate polarizing filters. Thus, loss of light occurs in the lenssystem, in addition to other problems such as the difficulty of makingthe apparatus more compact. The technology disclosed in JapaneseUnexamined Patent Application Publication No. 2004-309868 entails twopairs of a lens combined with a polarizing filter, and thus a morecomplex and bulky apparatus is difficult to avoid. Furthermore, usingsuch imaging apparatus to photograph ordinary 2D images rather than juststereo images creates additional apparatus complexity, and isimpractical.

In light of the foregoing, it is desirable to provide an imagingapparatus having a simple configuration and structure that is able toimage a subject as a stereo image, as well as an image sensor array usedin such an imaging apparatus.

According to a first, a second, and a third embodiments of the presentdisclosure to realize the above issues, there is provided an imagingapparatus including (A) an imaging lens and (B) an image sensor array inwhich a plurality of first image sensor units and a plurality of secondimage sensor units are arrayed. A single first image sensor unitincludes a single first microlens and a plurality of image sensors, asingle second image sensor unit includes a single second microlens and asingle image sensor, light passing through the imaging lens and reachingeach first image sensor unit passes through the first microlens andforms an image on the plurality of image sensors constituting the firstimage sensor unit, light passing through the imaging lens and reachingeach second image sensor unit passes through the second microlens andforms an image on the image sensor constituting the second image sensorunit, and an inter-unit light shielding layer is formed between theimage sensor units (specifically, at least between the first imagingunit and the second imaging unit, and between the second imaging unitsthemselves).

In addition, in the imaging apparatus according to the first embodimentof the present disclosure to realize the above issues, a light shieldinglayer is not formed between the image sensor units themselves whichconstitute the first image sensor unit. Further, in the imagingapparatus according to the second embodiment of the present disclosure,an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the first image sensorunit. Furthermore, in the imaging apparatus according to the thirdembodiment of the present disclosure, an inter-device light shieldinglayer is formed between the image sensor units themselves whichconstitute the first image sensor unit.

According to the first, the second, and the third embodiments of thepresent disclosure, there is provided an image sensor array including aplurality of first image sensor units, and a plurality of second imagesensor units arrayed therein. A single first image sensor unit includesa single first microlens and a plurality of image sensors, a singlesecond image sensor unit includes a single second microlens and a singleimage sensor, and an inter-unit light shielding layer is formed betweenthe image sensor units (specifically, at least between the first imagingunit and the second imaging unit, and between the second imaging unitsthemselves).

In addition, in an image sensor array according to the first embodimentof the present disclosure, a light shielding layer is not formed betweenthe image sensor units themselves which constitute the first imagesensor unit. Further, in an image sensor array according to the secondembodiment of the present disclosure, an inter-device light shieldinglayer is formed only partially between the image sensors themselveswhich constitute the first image sensor unit. Furthermore, in an imagesensor array according to the third embodiment of the presentdisclosure, an inter-device light shielding layer is formed between theimage sensor units themselves which constitute the first image sensorunit.

According to a fourth, a fifth, and a sixth embodiments of the presentdisclosure to realize the above issues, there is provided an imagingapparatus including (A) an imaging lens and (B) an image sensor array inwhich a plurality of image sensor units are arrayed. A single imagesensor unit includes a single microlens and a plurality of imagesensors, light passing through the imaging lens and reaching each imagesensor unit passes through the microlens and forms an image on theplurality of image sensors constituting the image sensor unit, and aninter-unit light shielding layer is formed between the image sensorunits themselves.

According to the fourth, the fifth, and the sixth embodiments of thepresent disclosure to realize the above issues, there is provided animage sensor array including a plurality of image sensor units arrayedtherein. A single image sensor unit includes a single microlens and aplurality of image sensors. An inter-unit light shielding layer isformed between the image sensor units themselves

In addition, in an imaging apparatus or an image sensor array accordingto the fourth embodiment of the present disclosure, an inter-devicelight shielding layer is formed between the image sensor unitsthemselves which constitute the image sensor unit. Further, in animaging apparatus or an image sensor array according to the fifthembodiment of the present disclosure, an inter-device light shieldinglayer is formed only partially between the image sensors themselveswhich constitute the image sensor unit. Furthermore, in an imagingapparatus or an image sensor array according to the sixth embodiment ofthe present disclosure, an inter-device light shielding layer is formedbetween the image sensor units themselves which constitute the imagesensor unit.

In an imaging apparatus or image sensor array according to the firstthrough third modes of the present disclosure, a single first imagesensor unit includes a single first microlens and multiple imagesensors, and image data for a stereo image can be obtained from such afirst image sensor unit, while in addition, in an imaging apparatus orimage sensor array according to the fourth through sixth modes of thepresent disclosure, a single image sensor unit includes a singlemicrolens and multiple image sensors, and image data for a stereo imagecan be obtained from such an image sensor unit. For this reason, theimaging apparatus does not become bulky, extremely precise positioningof the first microlens (or microlens) is not demanded, and the problemof a large drop in resolution does not occur. Furthermore, an imagesensor array can be manufactured without adding additional stages to themanufacturing process and without introducing special manufacturingprocesses. Moreover, a compact monocular imaging apparatus having asimple configuration and structure can be provided. Additionally, sincethe configuration does not involve two pairs of a lens combined with apolarizing filter, imbalances and discrepancies in factors such as zoom,aperture, focus, and angle of convergence do not occur. Moreover, sincethe baseline for binocular parallax is comparatively short, a naturalstereoscopic effect can be achieved. Also, 2D images and 3D images canbe easily obtained depending on how image data is processed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1(A) and 1(B) are, respectively, a schematic partial cross-sectionof an imaging apparatus or image sensor array according to Example 1,and a schematic illustration of how image sensors and microlenses aredisposed therein;

FIGS. 2(A) and 2(B) are graphs illustrating simulated results of therelationship between the sensitivity of image sensors constituting afirst image sensor unit and the angle of incidence for light incident onthe image sensors in an imaging apparatus or an image sensor arrayaccording to Example 1 and Example 2, respectively;

FIG. 3 is a conceptual illustration of light reaching an image sensorarray in an imaging apparatus according to Example 1;

FIGS. 4(A) and 4(B) are conceptual illustrations of light reaching animage sensor array in an imaging apparatus according to Example 1, whileFIGS. 4(C) and 4(D) schematically illustrate images formed on all firstimage sensor units in an image sensor array by the light illustrated inFIGS. 4(A) and 4(B);

FIG. 5(A) is a schematic illustration of a subject imaged by an imagingapparatus according to Example 1, while FIGS. 5(B) to 5(E) areconceptual illustrations of image data for a subject imaged by animaging apparatus according to Example 1;

FIG. 6 schematically illustrates how first image sensor units and secondimage sensor units are disposed in an image sensor array according toExample 1;

FIG. 7 is a schematic illustration highlighting how the first imagesensor units are disposed in the image sensor array according to Example1 illustrated in FIG. 6;

FIG. 8 is a schematic illustration highlighting how the second imagesensor units are disposed in the image sensor array according to Example1 illustrated in FIG. 6;

FIG. 9 is a conceptual illustration of an imaging apparatus according toExample 1;

FIG. 10 schematically illustrates how first image sensor units andsecond image sensor units are disposed in an exemplary modification ofan image sensor array according to Example 1;

FIG. 11 is a schematic illustration highlighting how the first imagesensor units are disposed in the exemplary modification of an imagesensor array according to Example 1 illustrated in FIG. 10;

FIG. 12 is a schematic illustration highlighting how the second imagesensor units are disposed in the exemplary modification of an imagesensor array according to Example 1 illustrated in FIG. 10;

FIG. 13 schematically illustrates how first image sensor units andsecond image sensor units are disposed in another exemplary modificationof an image sensor array according to Example 1;

FIG. 14 is a schematic illustration highlighting how the first imagesensor units are disposed in the other exemplary modification of animage sensor array according to Example 1 illustrated in FIG. 13;

FIG. 15 is a schematic illustration highlighting how the second imagesensor units are disposed in the other exemplary modification of animage sensor array according to Example 1 illustrated in FIG. 13;

FIGS. 16(A) and 16(B) are, respectively, a schematic partialcross-section of an imaging apparatus or image sensor array according toExample 2, and a schematic illustration of how image sensors andmicrolenses are disposed therein;

FIGS. 17(A) and 17(B) are, respectively, a schematic partialcross-section of an imaging apparatus or image sensor array according toExample 3, and a schematic illustration of how image sensors andmicrolenses are disposed therein;

FIGS. 18(A) and 18(B) are, respectively, a schematic partialcross-section of an imaging apparatus or image sensor array according toExample 4, and a schematic illustration of how image sensors andmicrolenses are disposed therein;

FIGS. 19(A) and 19(B) are, respectively, a schematic partialcross-section of an imaging apparatus or image sensor array according toExample 5, and a schematic illustration of how image sensors andmicrolenses are disposed therein;

FIGS. 20(A) and 20(B) are, respectively, a schematic partialcross-section of an imaging apparatus or image sensor array according toExample 6, and a schematic illustration of how image sensors andmicrolenses are disposed therein;

FIGS. 21(A) and 21(B) are, respectively, a schematic partialcross-section according to Example 7 (being an exemplary modification ofan imaging apparatus or image sensor array according to Example 4), anda schematic illustration of how image sensor and microlenses aredisposed therein;

FIGS. 22(A) and 22(B) are, respectively, a schematic partialcross-section according to Example 7 (being an exemplary modification ofan imaging apparatus or image sensor array according to Example 5), anda schematic illustration of how image sensors and microlenses aredisposed therein; and

FIGS. 23(A) and 23(B) are, respectively, a schematic partialcross-section according to Example 7 (being an exemplary modification ofan imaging apparatus or image sensor array according to Example 6), anda schematic illustration of how image sensors and microlenses aredisposed therein.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Hereinafter, present disclosure will be described on the basis ofexamples, but the present disclosure is not limited to the examples, andthe various numerical values and materials in the examples are forillustrative purposes. The description will proceed in the followingorder.

1. General description of imaging apparatus and image sensor arrayaccording to the first through sixth modes of the present disclosure

2. Example 1 (imaging apparatus and image sensor array according to thefirst mode of the present disclosure)

3. Example 2 (imaging apparatus and image sensor array according to thesecond mode of the present disclosure)

4. Example 3 (imaging apparatus and image sensor array according to thethird mode of the present disclosure)

5. Example 4 (imaging apparatus and image sensor array according to thefourth mode of the present disclosure)

6. Example 5 (imaging apparatus and image sensor array according to thefifth mode of the present disclosure)

7. Example 6 (imaging apparatus and image sensor array according to thesixth mode of the present disclosure)

8. Example 7 (modifications of Examples 4 to 6) and other

[General Description of Imaging Apparatus and Image Sensor ArrayAccording to the First through Sixth Modes of the Present Disclosure]

In the following description, a first image sensor unit in an imagingapparatus and image sensor array according to the first through thirdmodes of the present disclosure and an image sensor unit in an imagingapparatus and image sensor array according to the fourth through sixthmodes of the present disclosure may be collectively designated the“(first) image sensor unit” in some cases. Likewise, a first microlensin an imaging apparatus and image sensor array according to the firstthrough third modes of the present disclosure and a microlens in animaging apparatus and image sensor array according to the fourth throughsixth modes of the present disclosure may be collectively designated the“(first) microlens” in some cases.

In an imaging apparatus or image sensor array according to the firstthrough sixth modes of the present disclosure,

a (first) image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor,

which may be embodied such that the first image sensor and the secondimage sensor are disposed along a first direction, the third imagesensor is adjacent to the first image sensor along a second directionorthogonal to the first direction, and the fourth image sensor isadjacent to the second image sensor along the second direction.

Alternatively, in an imaging apparatus or image sensor array accordingto the second mode or the fifth mode of the present disclosure,

a (first) image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor,

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, and

an inter-device light shielding layer disposed in the boundary regionsof the first image sensor, the second image sensor, the third imagesensor, and the fourth image sensor. The planar shape of theinter-device light shielding layer may take a square pattern, but insome cases may also take a rectangular pattern that is elongated in thefirst direction. In the case where the planar shape of the inter-devicelight shielding layer is taken to be square, the length of an edge ofthe square is preferably from 0.1 to 0.2 times the length of an edge ofan image sensor constituting the (first) image sensor unit. In the casewhere the planar shape of the inter-device light shielding layer istaken to be rectangular, the length of the long edge is preferably from0.1 to 2 times the length of an edge of an image sensor constituting the(first) image sensor unit.

Furthermore, in the various preferable embodiments described above foran imaging apparatus or image sensor array according to the firstthrough sixth modes of the present disclosure,

take a first virtual plane to be the plane extending in the firstdirection that includes the normal line of the (first) microlens passingthrough the center of the (first) microlens, and take the normal line ofthe photosensitive surface of a photoelectric transducer discussed laterto be the base of reference for the value of a center angle of incidenceθ₀ for all (first) image sensor units obtained by collimated lightparallel to the first virtual plane from the spatial region above thefirst image sensor and the third image sensor that passes through the(first) microlens and reaches the second image sensor and the fourthimage sensor. In this case, the value of the center angle of incidenceθ₀ satisfies:0°≤θ₀≤15°Meanwhile, if θ_(n) is taken to be the angle of incidence and (Sθ_(n))is taken to be the received light intensity at an image sensor alongeach angle of incidence θ_(n), the center angle of incidence θ₀ can beexpressed as:θ₀=Σ{(θ_(n)×(Sθ _(n)))/(Sθ _(n))}

Alternatively, the baseline for binocular parallax between an imageobtained from the first image sensor and the third image sensor versusan image obtained from the second image sensor and the fourth imagesensor in all (first) image sensor units may be computed as 4f/(3πF),where f is the focal length of the imaging lens (in millimeters, forexample) and F is the f-number.

Additionally, in an imaging apparatus or an image sensor array accordingto the first through third modes of the present disclosure that includesthe preferable embodiments and configurations described above, a 1-pixelunit may be made up of multiple second image sensor units, andconfigured such that the surface area (or planar shape) occupied by asingle first image sensor unit is equivalent to (or in a similar shapeas) the surface area (or planar shape) occupied by a 1-pixel unit. Also,in this case, the width of an inter-unit light shielding layer in thefirst image sensor units may be configured to be greater than the widthof the inter-unit light shielding layer in the second image sensorunits. Thus, the amount of light bleed into the first image sensor unitsfrom the second image sensor units may be decreased. Furthermore, theinter-unit light shielding layers preferably satisfy0.1≤W ₂ /W ₁≤0.9where W₁ is the width of the inter-unit light shielding layer in thefirst image sensor units, and W₂ is the width of the inter-unit lightshielding layer in the second image sensor units. Also, in an imagingapparatus or an image sensor array according to the first through thirdmodes of the present disclosure that includes these configurations, itmay be configured such that the first image sensor unit includes fourimage sensors, and four second image sensor units constitute a 1-pixelunit.

In an imaging apparatus according to the first through third modes ofthe present disclosure, image data obtained from the first image sensorunits can be combined with image data obtained from the second imagesensor units to obtain data for a stereo image (right-eye image data andleft-eye image data). Specifically, a stereo image may be obtained onthe basis of image data obtained from the first and third image sensors(herein referred to as “first image data” for convenience), and imagedata obtained from the second and fourth image sensors (herein referredto as “second image data” for convenience) in all first image sensorunits. Alternatively, depending on whether the imaging apparatus isarranged horizontally or vertically, a stereo image may be obtained onthe basis of image data obtained from the first and second imagesensors, and image data obtained from the third and fourth imagesensors. The obtained stereo image may be based on the baseline forbinocular parallax discussed earlier, for example. More specifically, adepth map (depth information) may be created on the basis of first imagedata and second image data, for example, and data for a stereo image(right-eye image data and left-eye image data) may be obtained from thedepth map (depth information) and image data obtained from all secondimage sensor units. Note that the first image sensor units could also bereferred to as “image sensor units for parallax detection”, whereas thesecond image sensor units could also be referred to as “image sensorunits for image generation”.

In an imaging apparatus according to the fourth through sixth modes ofthe present disclosure, data for a stereo image (right-eye image dataand left-eye image data) may be obtained on the basis of image dataobtained from image sensor units. Specifically, a stereo image may beobtained on the basis of image data obtained from the first and thirdimage sensors (first image data), and image data obtained from thesecond and fourth image sensors (second image data) in all image sensorunits. Alternatively, depending on whether the imaging apparatus isarranged horizontally or vertically, a stereo image may be obtained onthe basis of image data obtained from the first and second imagesensors, and image data obtained from the third and fourth imagesensors. Alternatively, a stereo image may be obtained on the basis ofimage data obtained from the first image sensor (first image data),image data obtained from the third image sensor (first image data), andimage data obtained by processing image data obtained from the fourthimage sensor (first image data), as well as image data obtained from thesecond image sensor (second image data), image data obtained from thefourth image sensor (second image data), and image data obtained byprocessing image data obtained from the first image sensor (second imagedata) in all image sensors. The obtained stereo image may be based onthe baseline for binocular parallax discussed earlier, for example. Notethat in this case, the image sensors are not only image sensor units forparallax detection, but also image sensor units for image generation.

In an imaging apparatus or image sensor array according to the firstthrough third modes of the present disclosure, the first image sensorunits may be disposed on the grid points of a lattice in a firstdirection and a second direction. In other words, it may be configuredsuch that a single first image sensor unit is disposed in place of everyNth pixel unit (where N≥2) along a first direction, and in addition, asingle first image sensor unit is disposed in place of every Mth pixelunits (where M≥2, for example) along a second direction. Alternatively,it may be configured such that lines of first image sensor units aredisposed in a lattice in a first direction and a second direction. Inother words, it may be configured such that first image sensor units aredisposed along the entirety of a row extending in a first direction,with one row of first image sensor units being disposed with respect to(M−1) pixel units (where M≥2, for example) along a second direction,while in addition, first image sensor units are disposed along theentirety of a column extending in the second direction, with one columnof first image sensor units being disposed with respect to (N−1) pixelunits (where N≥2, for example) along the first direction. Alternatively,it may be configured such that first image sensor units are disposedalong the entirety of a row extending in a first direction, with one rowof first image sensor units being disposed with respect to (M−1) pixelunits (where M≥2, for example) along a second direction.

In an imaging apparatus or image sensor array according to the firstthrough sixth modes of the present disclosure including the variouspreferable embodiments and configurations described above (hereinafter,these may be simply and collectively referred to as “the presentdisclosure” in some cases), the first direction may correspond to thehorizontal direction of the image while the second direction correspondsto the vertical direction of the image in some cases, but the firstdirection may also correspond to the vertical direction of the imagewhile the second direction corresponds to the horizontal direction ofthe image in other cases, depending on the arrangement of the imagingapparatus, or in other words, depending on whether the imaging apparatusis arranged horizontally or vertically. The question of whether theimaging apparatus is arranged horizontally or vertically may be detectedby providing the imaging apparatus with an orientation detectingapparatus such as an acceleration sensor or a gyro sensor, for example,with the first direction and the second direction being determinedappropriately on the basis of the detection results.

In the present disclosure, an image sensor constituting a second imagesensor unit is realized by a photoelectric transducer as well as a colorfilter and a second microlens (on-chip lens) stacked on top of or abovethe photoelectric transducer. Alternatively, the image sensor may berealized by a photoelectric transducer as well as a second microlens(on-chip lens) and a color filter stacked on top of or above thephotoelectric transducer. Regarding all points other than the lightshielding layer, the configuration and structure of the image sensorsconstituting a (first) image sensor unit (the first image sensor, secondimage sensor, third image sensor, and fourth image sensor) may take thesame configuration and structure as the image sensor constituting asecond image sensor unit, and may furthermore take the configuration andstructure of an image sensor constituting a second image sensor unitwith the color filter removed, with the exception of the (first)microlens being formed instead of the second microlens (on-chip lens).Note that in the case of providing the first image sensor, second imagesensor, third image sensor, and fourth image sensor with color filters,a color filter that transmits red light, a color filter that transmitsgreen light, and a color filter that transmits blue light may beprovided, or alternatively, a color filter that transmits light of asingle color such as a color filter that transmits green light, forexample, may be provided. Alternatively, a transparent layer may beformed or a neutral density filter may be provided instead of providingthe first image sensor, second image sensor, third image sensor, andfourth image sensor with color filters.

A 1-pixel unit including multiple second image sensor units may bearranged in a Bayer array, for example. When using a Bayer array, each1-pixel unit includes one red image sensor sensitive to red light, oneblue image sensor sensitive to blue light, and two green image sensorssensitive to green light. However, the array of 1-pixel units made up ofmultiple second image sensor units is not limited to a Bayer array.Other potential arrays include interline arrays, G stripe RB mosaicarrays, G stripe RB full mosaic arrays, complementary mosaic arrays,stripe arrays, diagonal stripe arrays, primary chroma arrays, fieldsequential chroma arrays, frame sequential chroma arrays, MOS arrays,improved MOS arrays, frame-interleaved arrays, and field-interleavedarrays.

The imaging lens may be a fixed focal length lens, but may also be azoom lens. The configuration and structure of the lens or lens systemmay be determined on the basis of the specifications demanded of theimaging lens. Potential image sensors include CCD sensors, CMOS sensors,and charge modulation device (CMD) signal amplifying image sensors. Inaddition, the imaging apparatus may be a front-illuminated solid-stateimaging apparatus or a back-illuminated solid-state imaging apparatus.Furthermore, the imaging apparatus may constitute part of a digitalstill camera or video camera, camcorder, or a mobile phone with abuilt-in camera, commonly referred to as a camera phone.

Example 1

Example 1 relates to an imaging apparatus and image sensor arrayaccording to the first mode of the present disclosure. FIG. 1(A)illustrates a schematic partial cross-section of an imaging apparatusand image sensor array according to Example 1, while FIG. 1(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 1(A) is a schematic partial cross-section view taken alongthe line A in FIG. 1(B). Also, FIG. 2(A) illustrates simulated resultsof the relationship between the sensitivity of image sensorsconstituting a first image sensor unit and the angle of incidence forincident light in an imaging apparatus and image sensor array accordingto Example 1. FIGS. 3, 4(A), and 4(B) conceptually illustrate lightreaching an image sensor array in an imaging apparatus according toExample 1, while FIGS. 4(C) and 4(D) schematically illustrate imagesimaging formed on an image sensor array by the light illustrated inFIGS. 4(A) and 4(B). Also, FIG. 5(A) schematically illustrates a subjectimaged by an imaging apparatus according to Example 1, while FIGS. 5(B)to 5(E) conceptually illustrate image data for a subject imaged by animaging apparatus according to Example 1.

Additionally, FIG. 6 schematically illustrates how first image sensorunits and second image sensor units are disposed in an image sensorarray, while FIG. 7 is a schematic illustration highlighting how thefirst image sensor units are disposed in the image sensor arrayillustrated in FIG. 6, and FIG. 8 is a schematic illustrationhighlighting how the second image sensor units are disposed in the imagesensor array illustrated in FIG. 6. FIG. 9 is a conceptual illustrationof an imaging apparatus. Note that as a general rule in the followingdescription, the first direction corresponds to the horizontal directionof the image and the second direction corresponds to the verticaldirection of the image. For the sake of convenience, the first directionis taken to be the X direction and the second direction the Y direction,while the travel direction of light is taken to be the Z direction.

An imaging apparatus 10 according to Example 1 or Examples 2 to 3 laterdiscussed includes

(A) an imaging lens 20, and

(B) an image sensor array 30 in which multiple first image sensor units40 and multiple second image sensor units 50 are arrayed,

wherein a single first image sensor unit 40 includes a single firstmicrolens 45 and multiple image sensors, and

a single second image sensor unit 50 includes a single second microlens55 and a single image sensor 51. Also,

light passing through the imaging lens 20 and reaching each first imagesensor unit 40 passes through the first microlens 45 and forms an imageon the multiple image sensors constituting the first image sensor unit40, and

light passing through the imaging lens 20 and reaching each second imagesensor unit 50 passes through the second microlens 55 and forms an imageon the image sensor 51 constituting the second image sensor unit 50.Furthermore, inter-unit light shielding layers 46 and 56 are formedbetween image sensor units (specifically, at least between the firstimage sensor units 40 and the second image sensor units 50 and betweenthe second image sensor unit 50 themselves, and more specifically,between the first image sensor units 40 and the second image sensorunits 50 and between the second image sensor units 50 themselves in thisExample).

Note that in FIG. 1(B), image sensor boundaries are indicated by solidlines, and the inter-unit light shielding layers 46 and 56 are shaded.However, the inter-unit light shielding layer 56 is shaded in onelocation only. Also, the photoelectric transducer 61 later discussed isillustrated as a square shape, while the first microlens 45 and thesecond microlens 55 are illustrated as circular shapes, and the 1-pixelunit 50A is illustrated as a double-lined square. FIGS. 16(B), 17(B),18(B), 19(B), and 20(B) are illustrated similarly.

Also, an image sensor array according to Example 1 or Examples 2 to 3later discussed is

an image sensor array 30 in which multiple first image sensor units 40and multiple second image sensor units 50 are arrayed,

wherein a single first image sensor unit 40 includes a single firstmicrolens 45 and multiple image sensors,

a single second image sensor unit 50 includes a single second microlens55 and a single image sensor 51, and

inter-unit light shielding layers 46 and 56 are formed between imagesensor units (specifically, at least between the first image sensorunits 40 and the second image sensor units 50 and between the secondimage sensor unit 50 themselves, and more specifically, between thefirst image sensor units 40 and the second image sensor units 50 andbetween the second image sensor units 50 themselves in this Example).

In addition, in the imaging apparatus 10 or the image sensor array 30according to Example 1, a light shielding layer is not formed betweenthe image sensors themselves which constitute the first image sensorunit 40.

In the imaging apparatus 10 according to Example 1 or Examples 2 to 6later discussed, the imaging lens 20 causes light to form an image onthe image sensor array 30. The image sensor array 30 is disposed insidea main camera unit 11. The imaging apparatus may constitute part of adigital still camera or video camera, for example.

Besides the image sensor array 30, the main camera unit 11 is equippedwith an image processor 12 and image storage 13, for example.Additionally, right-eye image data and left-eye image data is formed onthe basis of electrical signals converted by the image sensor array 30.The image sensor array 30 may be realized with a charge-coupled device(CCD) or complementary metal-oxide-semiconductor (CMOS) image sensor,for example. The image processor 12 converts electrical signals outputfrom the image sensor array 30 into right-eye image data and left-eyeimage data, which is recorded to the image storage 13.

In Example 1 or Examples 2 to 3 later discussed, the first image sensorunit 40 includes four image sensors, these being a first image sensor41, a second image sensor 42, a third image sensor 43, and a fourthimage sensor 44. Also, the first image sensor 41 and the second imagesensor 42 are disposed along a first direction (X direction), the thirdimage sensor 43 is adjacent to the first image sensor 41 along a seconddirection (Y direction) orthogonal to the first direction, and thefourth image sensor 44 is adjacent to the second image sensor 42 alongthe second direction.

In Example 1 or Examples 2 to 6 later discussed, the first image sensor41, the second image sensor 42, the third image sensor 43, and thefourth image sensor 44 are disproportionately disposed towards thecenter of the first image sensor unit 40 (or the image sensor unit 140).Stated differently, the centers of the image sensors 41, 42, 43, and 44are not aligned with the centers of the photoelectric transducers 61respectively constituting the image sensors 41, 42, 43, and 44, suchthat the distance from the centers of the photoelectric transducers 61respectively constituting the image sensors 41, 42, 43, and 44 to thecenter of the first image sensor unit 40 (or the image sensor unit 140)is shorter than the distance from the centers of the image sensors 41,42, 43, and 44 to the center of the first image sensor unit 40 (or theimage sensor unit 140). In contrast, the center of the image sensor 51is aligned with the center of the second image sensor unit 50.

Also, in Example 1 or Examples 2 to 3 later discussed, a 1-pixel unit50A includes multiple second image sensor units 50, such that thesurface area (or planar shape) occupied by a single first image sensorunit 40 is equivalent to (or in a similar shape as) the surface area (orplanar shape) occupied by a 1-pixel unit 50A. Additionally, the width ofthe inter-unit light shielding layer 46 in the first image sensor unit40 is greater than the width of the inter-unit light shielding layer 56in the second image sensor unit 50. Although not limited thereto, theratio of the widths of the inter-unit light shielding layers 46 and 56is herein taken to be W₂/W₁=0.15, where W₁ is the width of theinter-unit light shielding layer 46 in the first image sensor unit 40,and W₂ is the width of the inter-unit light shielding layer 56 in thesecond image sensor unit 50. Furthermore, as discussed earlier, thefirst image sensor unit 40 includes four image sensors, and four secondimage sensor units 50 constitute a 1-pixel unit 50A. Note that the valueof the width W₁ in Example 2 or Example 3 later discussed is the samevalue as in Example 1.

Specifically, in this Example, the 1-pixel unit 50A including multiplesecond image sensor units 50 is arranged in a Bayer array. The 1-pixelunit 50A includes one red image sensor 51R sensitive to red light (insome cases labeled “R” in the drawings), two green image sensors 51Gsensitive to green light (in some cases labeled “G” in the drawings),and one blue image sensor 51B (in some cases labeled “B” in thedrawings).

The red image sensor 51R constituting the second image sensor unit 50 isrealized by a photoelectric transducer 61, as well as a firstinter-layer insulating layer 62, a color filter 63R, a secondinter-layer insulating layer 64, and a second microlens (on-chip lens)55 stacked on top of the photoelectric transducer 61. The green imagesensor 51G constituting the second image sensor unit 50 is realized by aphotoelectric transducer 61, as well as a first inter-layer insulatinglayer 62, a color filter 63G a second inter-layer insulating layer 64,and a second microlens (on-chip lens) 55 stacked on top of thephotoelectric transducer 61. The blue image sensor 51B constituting thesecond image sensor unit 50 is realized by a photoelectric transducer61, as well as a first inter-layer insulating layer 62, a color filter63B (not illustrated), a second inter-layer insulating layer 64, and asecond microlens (on-chip lens) 55 stacked on top of the photoelectrictransducer 61. An inter-unit light shielding layer 56 is formed betweenthe second image sensor units 50 themselves. The photoelectrictransducers 61 may be provided on a silicon semiconductor substrate 60,for example. Traces (not illustrated) are formed underneath thephotoelectric transducers 61.

Meanwhile, the image sensors constituting the first image sensor unit 40(the first image sensor 41, second image sensor 42, third image sensor43, and fourth image sensor 44) have a similar structure as an imagesensor constituting a second image sensor unit 50, with the exception ofa first microlens 45 being formed instead a second microlens (on-chiplens) 55, and a color filter 63G′ that transmits green light beingprovided for the first image sensor 41, second image sensor 42, thirdimage sensor 43, and fourth image sensor 44. Specifically, the imagesensors 41, 42, 43, and 44 constituting the first image sensor unit 40are realized by a photoelectric transducer 61, as well as a firstinter-layer insulating layer 62, a color filter 63G′, a secondinter-layer insulating layer 64, and a first microlens 45 stacked on topof the photoelectric transducer 61. The first microlens 45 covers thefour image sensors 41, 42, 43, and 44. The second microlens 55 is notformed on the image sensors constituting the first image sensor unit 40.The first microlens 45 and the second microlens 55 are formed on thesecond inter-layer insulating layer 64. The photoelectric transducers 61in the image sensors 41, 42, 43, and 44 constituting the first imagesensor unit 40 may be the same size or smaller than the photoelectrictransducer 61 in the image sensor 51 constituting the second imagesensor unit 50.

In the imaging apparatus 10 or image sensor array 30 according toExample 1, first image sensor units 40 may be disposed on the gridpoints of a lattice in a first direction and a second direction. Inother words, a single first image sensor unit 40 is disposed in place ofevery Nth 1-pixel unit 50A along a first direction (where N=2^(n) with nbeing a natural number from 1 to 5, for example; in the exampleillustrated in the drawings, n=2), and in addition, a single first imagesensor unit 40 is disposed in place of every Mth 1-pixel unit 50A alonga second direction (where M=2^(m) with m being a natural number from 1to 5, for example; in the example illustrated in the drawings, m=2).

Meanwhile, the graph in FIG. 2(A) illustrates simulated results of therelationship between the sensitivity of the image sensors 41, 42, 43,and 44 constituting the first image sensor unit 40 and the angle ofincidence for incident light in the imaging apparatus 10 or image sensorarray 30 according to Example 1. In FIGS. 2(A) and 2(B), the horizontalaxis represents the angle of incidence θ (in units of degrees) for lightincident on the microlenses, while the vertical axis represents thesensitivity (in arbitrary units). In FIGS. 2(A) and 2(B) herein, take afirst virtual plane to be the plane extending in the first directionthat includes the normal line of the first microlens 45 passing throughthe center of the first microlens 45 (i.e., the plane parallel to theplane of the page in FIG. 1(A) and perpendicular to the plane of thepage in FIG. 1(B)). In this case, the angle of incidence θ takes anegative value when collimated light parallel to the first virtual planefrom the spatial region above the first image sensor 41 and the thirdimage sensor 43 passes through the first microlens 45 and reaches thesecond image sensor 42 and the fourth image sensor 44.

Herein, the curve A in FIG. 2(A) illustrates the sensitivitycharacteristics of the second image sensor 42 and the fourth imagesensor 44, the curve B illustrates the sensitivity characteristics ofthe first image sensor 41 and the third image sensor 43, and the curve Cillustrates the sensitivity characteristics of the green image sensor51G FIG. 2(A) demonstrates that in the case where the value of the angleof incidence θ is negative, the sensitivity of the second image sensor42 and the fourth image sensor 44 is high, while the sensitivity of thefirst image sensor 41 and the third image sensor 43 is low. Conversely,in the case where the value of the angle of incidence θ is positive, thesensitivity of the second image sensor 42 and the fourth image sensor 44is low, while the sensitivity of the first image sensor 41 and the thirdimage sensor 43 is high. Consequently, first image data is generated byrays with a positive angle of incidence, or in other words by the firstimage sensor 41 and the third image sensor 43, whereas second image datais generated by rays with a negative angle of incidence, or in otherwords by the second image sensor 42 and the fourth image sensor 44. Inthis way, by disposing four image sensors 41, 42, 43, and 44 in thefirst image sensor unit 40 according to Example 1, a depth map forobtaining two sets of image data (right-eye image data and left-eyeimage data) can be created on the basis of first image data and secondimage data from all first image sensor units 40. Electrical signals fromthese image sensors 41, 42, 43, and 44 are then output simultaneously orin an alternating time series, and the output electrical signals (i.e.,the electrical signals output from the first image sensor units 40 andthe second image sensor units 50 in the image sensor array 30) areprocessed by the image processor 12 and recorded to the image storage 13as right-eye image data and left-eye image data.

FIG. 3 is a schematic illustration of light reaching the image sensorarray 30 in the imaging apparatus 10 according to Example 1 from theperspective of the imaging apparatus as a whole. The light indicated inbroken lines corresponds to light incident on the right eye, while thelight indicated in solid lines corresponds to light incident on the lefteye. Such light indicated in broken lines that corresponds to lightincident on the right eye is almost entirely received by the first imagesensor 41 and the third image sensor 43 in the first image sensor units40. Meanwhile, the light indicated in solid lines that corresponds tolight incident on the left eye is almost entirely received by the secondimage sensor 42 and the fourth image sensor 44 in the first image sensorunits 40. Thus, image data separated into first image data and secondimage data used to generated right-eye image data and left-eye imagedata can be obtained from all first image sensor units 40, even thoughthe image sensor array 30 has a simple configuration and structure.

FIG. 2(A) demonstrates that the value of the center angle of incidenceθ₀ (in degrees) obtained by collimated light parallel to the firstvirtual plane from the spatial region above the first image sensor 41and the third image sensor 43 passing through the first microlens 45 andreaching the second image sensor 42 and the fourth image sensor 44 inall first image sensor units 40 is θ₀=3.5° in the case where F=2.8.Also, the baseline for binocular parallax between an image obtained fromthe first image sensor 41 and the third image sensor 43 and an imageobtained from the second image sensor 42 and the fourth image sensor 44in all first image sensor units 40 is 4.2 mm in the case where f=35 mmand F=2.8.

Assume that the imaging lens 20 is focused on an object A with a squareshape, as schematically illustrated in FIGS. 4(A) and 4(B). Also assumethat an object B with a round shape is disposed closer to the imaginglens 20 than the object A. An image of the square object A is formed infocus on the image sensor array 30. Also, an image of the round object Bis formed out of focus on the image sensor array 30. Additionally, inthe example illustrated in FIG. 4(A), an image of the object B is formedon the image sensor array 30 at a position displaced to the right of theobject A by a distance +ΔX. Meanwhile, in the example illustrated inFIG. 4(B), an image of the object B is formed on the image sensor array30 at a position displaced to the left of the object A by a distance−ΔX. Consequently, the distance 2ΔX becomes information regarding thedepth of the object B. In other words, the amount and direction of thebokeh of an object positioned closer to the imaging apparatus 10 thanthe object A differs from the amount and direction of the bokeh of anobject positioned farther away from the imaging apparatus 10, and theamount of bokeh of the object B differs depending on the distancebetween the object A and the object B. Thus, a stereo image with thebaseline for binocular parallax discussed above can be obtained. Inother words, image data obtained from all first image sensor units 40[i.e., first image data (see the schematic diagram in FIG. 4(C)) andsecond image data (see the schematic diagram in FIG. 4(D))] can becombined with image data obtained from all second image sensor units 50in this way to obtain data for a stereo image (right-eye image data andleft-eye image data). In other words, a depth map (depth information)may be created on the basis of first image data obtained from the firstimage sensor 41 and the third image sensor 43 and second image dataobtained from the second image sensor 42 and the fourth image sensor 44in all first image sensor units 40, for example, and data for a stereoimage (right-eye image data and left-eye image data) may be obtainedfrom this depth map (depth information) and image data obtained from allsecond image sensor units 50. Note that such a method itself may be anestablished method in the related art.

FIG. 5(A) schematically illustrates subjects imaged by an imagingapparatus according to Example 1, while FIGS. 5(B) to 5(E) conceptuallyillustrate image data for subjects imaged by an imaging apparatusaccording to Example 1. The subjects imaged by the imaging apparatusaccording to Example 1 are taken to be bars extending in the seconddirection. As illustrated in FIG. 5(A), the subjects herein are threebars labeled A, B, and C. Bar A is positioned closest to the imagingapparatus, with bar B and C being successively positioned farther awayfrom the imaging apparatus. Also, assume that the bar B is in focus.Consequently, an image A_(C) of the bar A is in focus behind the imagesensor array 30, whereas an image C_(C) of the bar C is in focus infront of the image sensor array 30. In such a state, the image dataconceptually illustrated in FIG. 5B can be obtained in the second imagesensor units 50 of the image sensor array 30. Also, FIG. 5(C)conceptually illustrates image data (first image data) based on thefirst image sensor 41 and the third image sensor 43 in the first imagesensor units 40 of the image sensor array 30, with the images of thesubject bars A, B, and C being labeled A_(C), B_(R), and C_(R).Meanwhile, FIG. 5(D) conceptually illustrates image data (second imagedata) based on the second image sensor 42 and the fourth image sensor 44in the first image sensor units 40 of the image sensor array 30, withthe images of the subject bars A, B, and C being labeled A_(C), B_(L),and C_(L). Note that in FIGS. 5(C) and 5(D), the images B_(C) and C_(C)are also displayed in broken lines for reference. Furthermore, FIG. 5(E)conceptually illustrates image data superimposing the first image dataand the second image data. Herein, the image B_(R) and the image B_(L)of the bar B are separated by 2ΔB. Moreover, in FIG. 5(E), the imageB_(R) is positioned to the left while the image B_(L) is positioned tothe right. Likewise, the image C_(R) and the image C_(L) of the bar Care separated by 2ΔC. Moreover, in FIG. 5E, the image C_(L) ispositioned to the left while the image C_(R) is positioned to the right.In this way, by computing the positions and distances of images obtainedfrom the first image data and second image data, the relative positionalrelationships among in-focus subjects can be ascertained.

As above, in an imaging apparatus or image sensor array according toExample 1, a single first image sensor unit includes a single firstmicrolens and multiple image sensors, and image data for a stereo imagecan be obtained from all such first image sensor units. For this reason,the imaging apparatus does not become bulky, extremely precisepositioning of the first microlens is not demanded, and the problem of alarge drop in resolution does not occur. Furthermore, an image sensorarray can be manufactured without adding additional stages to themanufacturing process and without introducing special manufacturingprocesses. Moreover, a compact monocular imaging apparatus having asimple configuration and structure can be provided. Additionally, sincethe configuration does not involve two pairs of a lens combined with apolarizing filter, imbalances and discrepancies in factors such as zoom,aperture, focus, and angle of convergence do not occur. Also, since thebaseline for binocular parallax is comparatively short, natural stereoimages can be obtained, while 2D images and 3D images can be easilyobtained depending on how image data is processed.

However, instead of disposing the first image sensor units 40 on thegrid points of a lattice in a first direction and a second direction,lines of first image sensor units 40 may be disposed in a lattice in thefirst direction and the second direction, as illustrated in FIGS. 10 to12. FIG. 10 schematically illustrates how first image sensor units andsecond image sensor units may be disposed in an image sensor array. FIG.11 is a schematic illustration highlighting how the first image sensorunits may be disposed in the image sensor array, while FIG. 12 is aschematic illustration highlighting how the second image sensor unitsmay be disposed in the image sensor array. In other words, the firstimage sensor units 40 may be disposed along the entirety of a rowextending in the first direction, with one row of first image sensorunits 40 being disposed with respect to (M−1) 1-pixel units 50A alongthe second direction (where M=2^(m) with m being a natural number from 1to 5, for example; in the example illustrated in the drawings, m=2),while in addition, the first image sensor units 40 may be disposed alongthe entirety of a column extending in the second direction, with onecolumn of first image sensor units 40 being disposed with respect to(N−1) 1-pixel units 50A along the first direction (where N=2^(n) with nbeing a natural number from 1 to 5, for example; in the exampleillustrated in the drawings, n=2). Alternatively, the first image sensorunits 40 may be disposed as illustrated in FIGS. 13 to 15. FIG. 13schematically illustrates how first image sensor units and second imagesensor units may be disposed in an image sensor array. FIG. 14 is aschematic illustration highlighting how the first image sensor units maybe disposed in the image sensor array illustrated, while FIG. 15 is aschematic illustration highlighting how the second image sensor unitsmay be disposed in the image sensor array. In other words, the firstimage sensor units 40 may be disposed along the entirety of a rowextending in the first direction, with one row of first image sensorunits 40 being disposed with respect to (M−1) 1-pixel units 50A alongthe second direction (where M=2^(m) with m being a natural number from 1to 5, for example; in the example illustrated in the drawings, m=2).Note that such a configuration is preferably applied to an imagingapparatus that is only rarely ever arranged vertically, such as acamcorder, for example. Examples 2 to 3 later discussed may be similarlyconfigured. However, in such cases, the inter-unit light shieldinglayers 46 and 56 are also formed between the first image sensor units 40themselves, rather than only between the first image sensor units 40 andthe second image sensor units 50 and between the second image sensorunit 50 themselves.

Also, by providing the imaging apparatus with an orientation detectingapparatus such as an acceleration sensor or a gyro sensor, it can bedetected whether the imaging apparatus is arranged horizontally orarranged vertically, and on the basis of the detection results, it canbe determined whether to associate the first direction with thehorizontal direction of the image or with the vertical direction of theimage. Then, in the case where the first direction corresponds to thehorizontal direction of the image, first image data may be generatedfrom the first image sensor 41 and the third image sensor 43 whilesecond image data is generated from the second image sensor 42 and thefourth image sensor 44, as discussed above. Meanwhile, in the case wherethe first direction corresponds to the vertical direction of the image,first image data may be generated from the first image sensor 41 and thesecond image sensor 42 while second image data is generated from thethird image sensor 43 and the fourth image sensor 44, or alternatively,second image data may be generated from the first image sensor 41 andthe second image sensor 42 while first image data is generated from thethird image sensor 43 and the fourth image sensor 44.

Example 2

Example 2 relates to an imaging apparatus and image sensor arrayaccording to the second mode of the present disclosure. FIG. 16(A)illustrates a schematic partial cross-section of an imaging apparatus orimage sensor array according to Example 2, while FIG. 16(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 16(A) is a schematic partial cross-section view takenalong the line A in FIG. 16(B). In the imaging apparatus or image sensorarray according to Example 2, an inter-device light shielding layer 47is formed only partially between the image sensor themselves whichconstitute the first image sensor unit 40. Note that in FIG. 16(B),image sensor boundaries are indicated by solid lines, while theinter-unit light shielding layers 46 and 56 as well as the inter-devicelight shielding layer 47 are shaded. The inter-device light shieldinglayer 47 is positioned in the boundary area of the four image sensors(the first image sensor 41, second image sensor 42, third image sensor43, and fourth image sensor 44). In other words, the inter-device lightshielding layer 47 is positioned in the central area of the first imagesensor unit 40, and the planar shape of the inter-device light shieldinglayer 47 is square. Although not limited thereto, the length of an edgeof the square herein is specifically 0.2 times the length of an edge ofan image sensor constituting the first image sensor unit 40.

The graph in FIG. 2(B) illustrates simulated results of the relationshipbetween the sensitivity of image sensors constituting a first imagesensor unit and the angle of incidence for incident light in an imagingapparatus or image sensor array according to Example 2. The curve D inFIG. 2(B) illustrates the sensitivity characteristics of the secondimage sensor 42 and the fourth image sensor 44, while the curve Eillustrates the sensitivity characteristics of the first image sensor 41and the third image sensor 43. Note that the curves A and B illustratedin FIG. 2(A) are also illustrated in FIG. 2(B). Similarly to Example 1,FIG. 2(B) demonstrates that in the case where the value of the angle ofincidence θ is negative, the sensitivity of the second image sensor 42and the fourth image sensor 44 is high, while the sensitivity of thefirst image sensor 41 and the third image sensor 43 is low. Conversely,in the case where the value of the angle of incidence θ is positive, thesensitivity of the second image sensor 42 and the fourth image sensor 44is low, while the sensitivity of the first image sensor 41 and the thirdimage sensor 43 is high. Consequently, first image data is generated byrays with a positive angle of incidence, or in other words by the firstimage sensor 41 and the third image sensor 43, whereas second image datais generated by rays with a negative angle of incidence, or in otherwords by the second image sensor 42 and the fourth image sensor 44. Inthis way, by likewise disposing four image sensors 41, 42, 43, and 44 inthe first image sensor unit 40 in Example 2, a depth map for obtainingtwo sets of image data (right-eye image data and left-eye image data)can be created on the basis of first image data and second image datafrom all first image sensor units 40. Electrical signals from theseimage sensors 41, 42, 43, and 44 are then output simultaneously or in analternating time series, and the output electrical signals (i.e., theelectrical signals output from the first image sensor units 40 and thesecond image sensor units 50 in the image sensor array 30) are processedby the image processor 12 and recorded to the image storage 13 asright-eye image data and left-eye image data.

FIG. 2(B) demonstrates that the value of the center angle of incidenceθ₀ (in degrees) obtained by collimated light parallel to the firstvirtual plane from the spatial region above the first image sensor 41and the third image sensor 43 passing through the first microlens 45 andreaching the second image sensor 42 and the fourth image sensor 44 inall first image sensor units 40 is

θ₀=5.3°

in the case where F=2.8, an improvement over

θ₀=3.5°

in Example 1. Also, the baseline for binocular parallax between an imageobtained from the first image sensor 41 and the third image sensor 43and an image obtained from the second image sensor 42 and the fourthimage sensor 44 in all first image sensor units 40 is 6.4 mm in the casewhere f=35 mm and F=2.8, longer than the baseline in Example 1. In thisway, by providing an inter-device light shielding layer 47, the value ofthe center angle of incidence θ₀ is improved and the baseline forbinocular parallax is lengthened over Example 1. This is due to thecenter angle of incidence being shifted outward relatively as a resultof incident light from the pupil center being cut by the light shieldinglayer.

Except for the above point, the configuration and structure of theimaging apparatus or image sensor array according to Example 2 may betaken to be similar to the configuration and structure of the imagingapparatus or image sensor array described in Example 1, and for thisreason detailed description thereof will be omitted.

Example 3

Example 3 relates to an imaging apparatus and image sensor arrayaccording to the third mode of the present disclosure. FIG. 17(A)illustrates a schematic partial cross-section of an imaging apparatus orimage sensor array according to Example 3, while FIG. 17(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 17(A) is a schematic partial cross-section view takenalong the line A in FIG. 17(B). In the imaging apparatus or image sensorarray according to Example 3, an inter-device light shielding layer 48is formed between the image sensors themselves which constitute thefirst image sensor unit 40. In other words, the image sensor 41, theimage sensor 42, the image sensor 43, and the image sensor 44 arepartitioned from each other by the inter-device light shielding layer48, and the inter-device light shielding layer 48 is contiguous. Notethat in FIG. 17(B), image sensor boundaries are indicated by solidlines, while the inter-unit light shielding layers 46 and 56 as well asthe inter-device light shielding layer 48 are shaded. Specifically, thewidth of the inter-device light shielding layer 48 is preferably from0.05 to 0.02 times the length of an edge of an image sensor constitutingthe first image sensor unit 40, although not limited thereto, and isspecifically 0.1 in Example 3. Example 6 later discussed is similar.Similarly to Example 1, in the case where the value of the angle ofincidence θ is negative in Example 3, the sensitivity of the secondimage sensor 42 and the fourth image sensor 44 is high, while thesensitivity of the first image sensor 41 and the third image sensor 43is low. Conversely, in the case where the value of the angle ofincidence θ is positive, the sensitivity of the second image sensor 42and the fourth image sensor 44 is low, while the sensitivity of thefirst image sensor 41 and the third image sensor 43 is high.

Except for the above point, the configuration and structure of theimaging apparatus or image sensor array according to Example 3 may betaken to be similar to the configuration and structure of the imagingapparatus or image sensor array described in Example 1, and for thisreason detailed description thereof will be omitted. Likewise in Example3, the imaging apparatus does not become bulky, extremely precisepositioning of the first microlens is not demanded, and the problem of alarge drop in resolution does not occur. Furthermore, an image sensorarray can be manufactured without adding additional stages to themanufacturing process and without introducing special manufacturingprocesses.

Example 4

Example 4 relates to an imaging apparatus and image sensor arrayaccording to the fourth mode of the present disclosure. FIG. 18(A)illustrates a schematic partial cross-section of an imaging apparatusand image sensor array according to Example 4, while FIG. 18(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 18(A) is a schematic partial cross-section view takenalong the line A in FIG. 18(B).

An imaging apparatus 10 according to Example 4 or Examples 5 to 6 laterdiscussed includes

(A) an imaging lens 20, and

(B) an image sensor array 130 in which multiple image sensor units 140are arrayed,

wherein a single image sensor unit 140 includes a single microlens 145and multiple image sensors. Also, light passing through the imaging lens20 and reaching each image sensor unit 140 passes through the microlens145 and forms an image on the multiple image sensors constituting theimage sensor unit 140, while in addition, an inter-unit light shieldinglayer 146 is formed between the image sensor units 140 themselves.

Note that in FIG. 18(B) or FIG. 21(B) later discussed, the boundariesbetween the image sensors themselves are indicated by solid double lineswith a white fill, the boundaries of each image sensor are indicated bysolid lines, and the inter-unit light shielding layer 146 is shaded. Thephotoelectric transducers 61 are indicated by square shapes, while themicrolenses 145 are indicated by circular shapes. FIGS. 19(B), 20(B),21(B), 22(B), and 23(B) are illustrated similarly.

Also, an image sensor array according to Example 4 or Examples 5 to 6later discussed is

an image sensor array 130 in which multiple image sensor units 140 arearrayed,

wherein a single image sensor unit 140 includes a single microlens 145and multiple image sensors, and

an inter-unit light shielding layer 146 is formed between the imagesensor units 140 themselves.

In addition, in the imaging apparatus 10 or the image sensor array 130according to Example 4, a light shielding layer is not formed betweenthe image sensors themselves which constitute the image sensor unit 140.Note that the value of the width W₁ of the inter-unit light shieldinglayer 146 in the image sensor unit 140 is the same value as inExample 1. The value of the width W₁ in Examples 5 to 7 later discussedis also the same value as in Example 1.

In Example 4 or Examples 5 to 6 later discussed, the image sensor unit140 includes four image sensors, these being a first image sensor 41, asecond image sensor 42, a third image sensor 43, and a fourth imagesensor 44. Also, the first image sensor 41 and the second image sensor42 are disposed along a first direction (X direction), the third imagesensor 43 is adjacent to the first image sensor 41 along a seconddirection (Y direction) orthogonal to the first direction, and thefourth image sensor 44 is adjacent to the second image sensor 42 alongthe second direction.

The configuration and structure of the image sensors constituting theimage sensor unit 140 (the first image sensor 41, second image sensor42, third image sensor 43, and fourth image sensor 44) may be taken tobe similar to that of the image sensors described in Example 1.Specifically, the image sensors 41, 42, 43, and 44 constituting theimage sensor unit 140 are realized by a photoelectric transducer 61, aswell as a first inter-layer insulating layer 62, a color filter 63G′, asecond inter-layer insulating layer 64, and a microlens 145 stacked ontop of the photoelectric transducer 61. The microlens 145 covers thefour image sensors 41, 42, 43, and 44.

In the imaging apparatus 10 or image sensor array 130 according toExample 4, image sensor units 140 may be disposed in a first directionand a second direction.

In Example 4, and similarly to that described in Example 1, first imagedata is generated by rays with a positive angle of incidence, or inother words by the first image sensor 41 and the third image sensor 43,whereas second image data is generated by rays with a negative angle ofincidence, or in other words by the second image sensor 42 and thefourth image sensor 44. In this way, by likewise disposing four imagesensors 41, 42, 43, and 44 in the image sensor unit 140 in Example 4,data for a stereo image can be created on the basis of first image dataand second image data from all image sensor units 140. Electricalsignals from these image sensors 41, 42, 43, and 44 are then outputsimultaneously or in an alternating time series, and the outputelectrical signals (i.e., the electrical signals output from the imagesensor units 140 in the image sensor array 130) are processed by theimage processor 12 and recorded to the image storage 13 as right-eyeimage data and left-eye image data. In other words, data for a stereoimage (right-eye image data and left-eye image data) may be obtained onthe basis of first image data obtained from the first image sensor 41and the third image sensor 43 and second image data obtained from thesecond image sensor 42 and the fourth image sensor 44 in all imagesensor units 140. Note that such a method itself may be an establishedmethod in the related art.

Similarly to Example 1, by providing the imaging apparatus with anorientation detecting apparatus such as an acceleration sensor or a gyrosensor, it can be detected whether the imaging apparatus is arrangedhorizontally or arranged vertically, and on the basis of the detectionresults, it can be determined whether to associate the first directionwith the horizontal direction of the image or with the verticaldirection of the image. Then, in the case where the first directioncorresponds to the horizontal direction of the image, first image datamay be generated from the first image sensor 41 and the third imagesensor 43 while second image data is generated from the second imagesensor 42 and the fourth image sensor 44, as discussed above. Meanwhile,in the case where the first direction corresponds to the verticaldirection of the image, first image data may be generated from the firstimage sensor 41 and the second image sensor 42 while second image datais generated from the third image sensor 43 and the fourth image sensor44, or alternatively, second image data may be generated from the firstimage sensor 41 and the second image sensor 42 while first image data isgenerated from the third image sensor 43 and the fourth image sensor 44.

As above, in an imaging apparatus or image sensor array according toExample 4, a single image sensor unit includes a single microlens andmultiple image sensors, and image data for a stereo image can beobtained from all such image sensor units. For this reason, the imagingapparatus does not become bulky, extremely precise positioning of themicrolens is not demanded, and the problem of a large drop in resolutiondoes not occur. Furthermore, an image sensor array can be manufacturedwithout adding additional stages to the manufacturing process andwithout introducing special manufacturing processes. Moreover, a compactmonocular imaging apparatus having a simple configuration and structurecan be provided. Additionally, since the configuration does not involvetwo pairs of a lens combined with a polarizing filter, imbalances anddiscrepancies in factors such as zoom, aperture, focus, and angle ofconvergence do not occur. Also, since the baseline for binocularparallax is comparatively short, natural stereo images can be obtained,while 2D images and 3D images can be easily obtained depending on howimage data is processed.

Example 5

Example 5 relates to an imaging apparatus and image sensor arrayaccording to the fifth mode of the present disclosure. FIG. 19(A)illustrates a schematic partial cross-section of an imaging apparatus orimage sensor array according to Example 5, while FIG. 19(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 19(A) is a schematic partial cross-section view takenalong the line A in FIG. 19(B). In the imaging apparatus or image sensorarray according to Example 5, an inter-device shielding layer 147 isformed only partially between the image sensors themselves whichconstitute the image sensor unit 140. Note that in FIG. 19(B) or FIG.22(B) discussed later, image sensor boundaries are indicated by solidlines, while the inter-unit light shielding layer 146 and theinter-device light shielding layer 147 are shaded. The inter-devicelight shielding layer 147 is positioned in the boundary area of the fourimage sensors (the first image sensor 41, second image sensor 42, thirdimage sensor 43, and fourth image sensor 44). In other words, theinter-device light shielding layer 147 is positioned in the central areaof the image sensor unit 140, and the planar shape of the inter-devicelight shielding layer 147 is square. Herein, the length of an edge ofthe square is taken to be similar to that of Example 2.

By likewise disposing four image sensors 41, 42, 43, and 44 in the imagesensor unit 140 in Example 5, data for a stereo image can be created onthe basis of first image data and second image data from all imagesensor units 140. Electrical signals from these image sensors 41, 42,43, and 44 are then output simultaneously or in an alternating timeseries, and the output electrical signals (i.e., the electrical signalsoutput from the image sensor units 140 in the image sensor array 130)are processed by the image processor 12 and recorded to the imagestorage 13 as right-eye image data and left-eye image data.

Except for the above point, the configuration and structure of theimaging apparatus or image sensor array according to Example 5 may betaken to be similar to the configuration and structure of the imagingapparatus or image sensor array described in Example 4, and for thisreason detailed description thereof will be omitted.

Example 6

Example 6 relates to an imaging apparatus and image sensor arrayaccording to the sixth mode of the present disclosure. FIG. 20(A)illustrates a schematic partial cross-section of an imaging apparatus orimage sensor array according to Example 6, while FIG. 20(B)schematically illustrates how image sensors and microlenses are disposedtherein. FIG. 20(A) is a schematic partial cross-section view takenalong the line A in FIG. 20(B). In the imaging apparatus or image sensorarray according to Example 6, an inter-device light shielding layer 148is formed between the image sensors themselves which constitute theimage sensor unit 140. In other words, the image sensor 41, the imagesensor 42, the image sensor 43, and the image sensor 44 are partitionedfrom each other by the inter-device light shielding layer 148, and theinter-device light shielding layer 148 is contiguous. Note that in FIG.20(B) or FIG. 23(B) discussed later, image sensor boundaries areindicated by solid lines, while the inter-unit light shielding layer 146and the inter-device light shielding layer 148 are shaded. The width ofthe inter-device light shielding layer 148 is specifically taken to besimilar to that of Example 3. Similarly to Example 4, in the case wherethe value of the angle of incidence θ is negative in Example 6, thesensitivity of the second image sensor 42 and the fourth image sensor 44is high, while the sensitivity of the first image sensor 41 and thethird image sensor 43 is low. Conversely, in the case where the value ofthe angle of incidence θ is positive, the sensitivity of the secondimage sensor 42 and the fourth image sensor 44 is low, while thesensitivity of the first image sensor 41 and the third image sensor 43is high.

Except for the above point, the configuration and structure of theimaging apparatus or image sensor array according to Example 6 may betaken to be similar to the configuration and structure of the imagingapparatus or image sensor array described in Example 4, and for thisreason detailed description thereof will be omitted. Likewise in Example6, the imaging apparatus does not become bulky, extremely precisepositioning of the first microlens is not demanded, and the problem of alarge drop in resolution does not occur. Furthermore, an image sensorarray can be manufactured without adding additional stages to themanufacturing process and without introducing special manufacturingprocesses.

Example 7

Example 7 is a modification of Examples 4 to 6. In Examples 4 to 6, theimage sensor unit 140 includes four image sensors, these being a firstimage sensor 41, a second image sensor 42, a third image sensor 43, anda fourth image sensor 44. In other words, the image sensors 41, 42, 43,and 44 are configured as image sensors sensitive to one color of light(specifically, green light). Meanwhile, in Example 7, an image sensorunit 240 includes four image sensors, these being a first image sensor41R, a second image sensor 42G, a third image sensor 43G, and a fourthimage sensor 44B.

FIG. 21(A) illustrates a schematic partial cross-section of amodification of an imaging apparatus and image sensor array according toExample 4, while FIG. 21(B) schematically illustrates how image sensorsand microlenses are disposed therein. Also, FIG. 22(A) illustrates aschematic partial cross-section of a modification of an imagingapparatus and image sensor array according to Example 5, while FIG.22(B) schematically illustrates how image sensors and microlenses aredisposed therein. Furthermore, FIG. 23(A) illustrates a schematicpartial cross-section of a modification of an imaging apparatus andimage sensor array according to Example 6, while FIG. 23(B)schematically illustrates how image sensors and microlenses are disposedtherein. Note that FIGS. 21(A), 22(A), and 23(A) are schematic partialcross-section views taken along the line A in FIGS. 21(B), 22(B), and23(B), respectively.

The configuration and structure of the first image sensor 41R are thesame as that of the red image sensor 51R constituting a second imagesensor unit 50 described in Example 1, while the configuration andstructure of the second image sensor 42G and the third image sensor 43Gare the same as those of the green image sensor 51G constituting asecond image sensor unit 50 described in Example 1, and theconfiguration and structure of the fourth image sensor 44B are the sameas those of the blue image sensor 51B constituting a second image sensorunit 50 described in Example 1. Specifically, the image sensors 41R,42G, 43G, and 44B constituting the image sensor unit 240 are realized bya photoelectric transducer 61, as well as a first inter-layer insulatinglayer 62, a color filter 63R, 63G, or 63B, a second inter-layerinsulating layer 64, and a microlens 145 stacked on top of thephotoelectric transducer 61. However, unlike the image sensors 51R, 51Gand 51B in Example 1, a single microlens 145 covers the four imagesensors 41R, 42G, 43G, and 44B. Also, similarly to Examples 4 to 6, thefirst image sensor 41R and the second image sensor 42G are disposedalong a first direction (X direction), the third image sensor 43G isadjacent to the first image sensor 41R along a second direction (Ydirection) orthogonal to the first direction, and the fourth imagesensor 44B is adjacent to the second image sensor 42G along the seconddirection.

An imaging apparatus 10 according to Example 7 includes

(A) an imaging lens 20, and

(B) an image sensor array 230 in which multiple image sensor units 240are arrayed,

wherein a single image sensor unit 240 includes a single microlens 145and multiple image sensors. Also,

light passing through the imaging lens 20 and reaching each image sensorunit 240 passes through the microlens 145 and forms an image on themultiple image sensors constituting the image sensor unit 240.

Also, an image sensor array according to Example 7 is an image sensorarray 230 in which multiple image sensor units 240 are arrayed, whereina single image sensor unit 240 includes a single microlens 145 andmultiple image sensors 41R, 42G. 43G, and 44B.

In addition, in the imaging apparatus 10 or image sensor array 230according to Example 7, image sensor units 240 may be disposed in afirst direction and a second direction.

Similarly to Example 4, an inter-unit light shielding layer 146 isformed between the image sensor units 240 themselves (see FIGS. 21(A)and 21(B)). Alternatively, similarly to Example 5, an inter-device lightshielding layer 147 is formed only partially between the image sensorsthemselves which constitute the image sensor unit 240 (see FIGS. 22(A)and 22(B)). Alternatively, similarly to Example 6, an inter-device lightshielding layer 148 is formed between the image sensors themselves whichconstitute the image sensor unit 240 (see FIGS. 23(A) and 23(B)).

In Example 7, and similarly to that described in Example 1, first imagedata is generated by rays with a positive angle of incidence, or inother words by the first image sensor 41R and the third image sensor43G; whereas second image data is generated by rays with a negativeangle of incidence, or in other words by the second image sensor 42G andthe fourth image sensor 44B. Specifically, red-related first image datais obtained by the first image sensor 41R, green-related first imagedata is obtained by the third image sensor 43G, green-related secondimage data is obtained by the second image sensor 42G, and blue-relatedsecond image data is obtained by the fourth image sensor 41B.Furthermore, blue-related first image data is obtained from image dataobtained by the third image sensor 43G, the second image sensor 42G; andthe fourth image sensor 44B. Also, red-related second image data isobtained from image data obtained by the third image sensor 43G, thesecond image sensor 42G, and the first image sensor 41R.

In this way, by likewise disposing four image sensors 41R, 42G, 43G, and44B in the image sensor unit 240 in Example 7, data for a stereo imagecan be created on the basis of first image data and second image datafrom all image sensor units 240. Electrical signals from these imagesensors 41R, 42G, 43G, and 44B are then output simultaneously or in analternating time series, and the output electrical signals (i.e., theelectrical signals output from the image sensor units 240 in the imagesensor array 230) are processed by the image processor 12 and recordedto the image storage 13 as right-eye image data and left-eye image data.In other words, data for a stereo image (right-eye image data andleft-eye image data) may be obtained on the basis of red- andgreen-related first image data obtained from the first image sensor 41Rand the third image sensor 43G, and blue-related first image dataobtained by processing image data obtained from the fourth image sensor44B, as well as green- and blue-related second image data obtained fromthe second image sensor 42G and the fourth image sensor 44B, andred-related second image data obtained by processing image data obtainedfrom the first image sensor 41R in all image sensor units 240. Such amethod itself may be an established method in the related art. Note thatin Example 7, the baseline for binocular parallax may be obtained on thebasis of an image obtained from the third image sensor 43G and an imageobtained from the second image sensor 42G

As above, in an imaging apparatus or image sensor array according toExample 7, a single image sensor unit includes a single microlens andmultiple image sensors, and image data for a stereo image can beobtained from all such image sensor units. For this reason, the imagingapparatus does not become bulky, extremely precise positioning of themicrolens is not demanded, and the problem of a large drop in resolutiondoes not occur. Furthermore, an image sensor array can be manufacturedwithout adding additional stages to the manufacturing process andwithout introducing special manufacturing processes. Moreover, a compactmonocular imaging apparatus having a simple configuration and structurecan be provided. Additionally, since the configuration does not involvetwo pairs of a lens combined with a polarizing filter, imbalances anddiscrepancies in factors such as zoom, aperture, focus, and angle ofconvergence do not occur. Also, since the baseline for binocularparallax is comparatively short, natural stereo images can be obtained,while 2D images and 3D images can be easily obtained depending on howimage data is processed.

The foregoing thus describes the present disclosure on the basis ofpreferred examples, but the present disclosure is not limited to theseexamples. The configuration and structure of the imaging apparatus,image sensors, image sensor arrays, and image sensor units described inthe examples are for illustrative purposes, and may be modified asappropriate. The image sensors may be taken to be back-illuminated asillustrated in the drawings, but may also be taken to befront-illuminated, although not illustrated in the drawings as such.

A stereo image is displayed on the basis of right-eye image data andleft-eye image data, and the technique for such display may involve alenticular lens, a parallax barrier, or other technique in whichcircularly-polarized or linearly-polarized filters are applied to twoprojectors, the right-eye and left-eye images are respectivelydisplayed, and an image is observed with circularly-polarized orlinearly-polarized glasses corresponding to the display, for example.Note that an ordinary 2D (flat) image can be observed if an image isobserved without using circularly-polarized or linearly-polarizedglasses. Also, the processing sequence described in the foregoing may beinterpreted as a method having a series of such operations, but may alsobe interpreted as a program for causing a computer to execute a seriesof such operations, or a recording medium storing such a program. Themedium used as the recording medium may be a Compact Disc (CD), MiniDisc(MD), Digital Versatile Disc (DVD), memory card, or Blu-ray Disc(registered trademark), for example.

An imaging apparatus according to the first to sixth modes of thepresent disclosure may also be applied to a focusing method thatcontrols the focusing of an imaging lens on the basis of image databased on electrical signals output from a first image sensor and a thirdimage sensor, and image data based on electrical signals output from asecond image sensor and a fourth image sensor. In other words, focus canbe determined on the basis of first image data and second image datafrom first image sensor units. Specifically, assume that when the userof the imaging apparatus selects a subject to bring into focus on thebasis of an established method, an image of that subject is formed onthe image sensor array 30 or 130 in an unfocused state. In this case,the distance 2ΔX on the image sensor array 30 or 130 becomes informationregarding the subject's focus, or in other words the magnitude anddirection of misalignment from the in-focus position, similarly to thatdescribed in FIG. 4. Consequently, the subject can be put into focus ifthe focusing of the imaging lens is controlled, or in other words if thefocus function of the imaging lens is made to operate, such that thedistance 2ΔX becomes 0. Note that depending on whether the imagingapparatus is arranged horizontally or vertically, first image data maybe generated on the basis of the first image sensor and the third imagesensor (or the first image sensor and the second image sensor), whilesecond image data may be generated on the basis of the second imagesensor and the fourth image sensor (or the third image sensor and thefourth image sensor). However, the configuration is not limited thereto,and a two-dimensional focus determination can also be made by generatingthird image data on the basis of the first image sensor and the secondimage sensor (i.e., image data similar to the first image data) whilegenerating fourth image data on the basis of the third image sensor andthe fourth image sensor (i.e., image data similar to the second imagedata), rather than just generating first image data on the basis of thefirst image sensor and the third image sensor while generating secondimage data on the basis of the second image sensor and the fourth imagesensor. Note that the first image data and the second image data (andadditionally, the third image data and the fourth image data) may beimage data obtained from all first image sensor units, but may also beimage data obtained from first image sensor units positioned in thevicinity of the subject to be brought into focus.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Additionally, the present technology may also be configured as below.

[1]«Imaging apparatus: first embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of first image sensorunits and a plurality of second image sensor units are arrayed,

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

light passing through the imaging lens and reaching each first imagesensor unit passes through the first microlens and forms an image on theplurality of image sensors constituting the first image sensor unit,

light passing through the imaging lens and reaching each second imagesensor unit passes through the second microlens and forms an image onthe image sensor constituting the second image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the first image sensor unit.

[2]«Imaging apparatus: second embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of first image sensorunits and a plurality of second image sensor units are arrayed,

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

light passing through the imaging lens and reaching each first imagesensor unit passes through the first microlens and forms an image on theplurality of image sensors constituting the first image sensor unit,

light passing through the imaging lens and reaching each second imagesensor unit passes through the second microlens and forms an image onthe image sensor constituting the second image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the first image sensorunit.

[3]«Imaging apparatus: third embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of first image sensorunits and a plurality of second image sensor units are arrayed,

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

light passing through the imaging lens and reaching each first imagesensor unit passes through the first microlens and forms an image on theplurality of image sensors constituting the first image sensor unit,

light passing through the imaging lens and reaching each second imagesensor unit passes through the second microlens and forms an image onthe image sensor constituting the second image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the first image sensor unit.

[4] The Imaging Apparatus According to any One of [1] to [3], wherein

the first image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction.

[5] The Imaging Apparatus According to [2], wherein

the first image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, and

the inter-device light shielding layer is disposed in the boundaryregions of the first image sensor, the second image sensor, the thirdimage sensor, and the fourth image sensor, and the planar shape of theinter-device light shielding layer is square.

[6] The Imaging Apparatus According to [4] or [5], wherein

provided that a first virtual plane is the plane extending in a firstdirection that includes the normal line of the first microlens passingthrough the center of the first microlens, the value of a center angleof incidence θ₀ obtained by collimated light parallel to the firstvirtual plane from the spatial region above the first image sensor andthe third image sensor passing through the first microlens and reachingthe second image sensor and the fourth image sensor in all first imagesensor units satisfies θ°≤θ₀≤15°.

[7] The Imaging Apparatus According to [4] or [5], wherein

a length of the baseline for binocular parallax between an imageobtained from the first image sensor and the third image sensor versusan image obtained from the second image sensor and the fourth imagesensor in all first image sensor units is computed as 4f/(3πF) (mm),where f is the focal length of the imaging lens and F is the f-number.

[8] The Imaging Apparatus According to any one of [1] to [7], wherein

a 1-pixel unit includes a plurality of second image sensor units, and

the surface area occupied by a single first image sensor unit is equalto the surface area occupied by a 1-pixel unit.

[9] The Imaging Apparatus According to [8], wherein

the width of the inter-unit light shielding layer in the first imagesensor unit is greater than the width of the inter-unit light shieldinglayer in the second image sensor unit.

[10] The Imaging Apparatus According to any One of [1] to [9], wherein

the first image sensor unit includes four image sensors, and

four second image sensor units constitute a 1-pixel unit.

[11]«Image Sensor Array: First Embodiment»

An image sensor array including:

a plurality of first image sensor units; and

a plurality of second image sensor units arrayed therein;

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the first image sensor unit.

[12]«Image Sensor Array: Second Embodiment»

An image sensor array including:

a plurality of first image sensor units; and

a plurality of second image sensor units arrayed therein;

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the first image sensorunit.

[13]«Image Sensor Array: Third Embodiment»

An image sensor array including:

a plurality of first image sensor units; and

a plurality of second image sensor units arrayed therein;

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the first image sensor unit.

[14]«Image Sensor Array: Fourth Embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of image sensor units arearrayed;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

light passing through the imaging lens and reaching each image sensorunit passes through the microlens and forms an image on the plurality ofimage sensors constituting the image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the image sensor unit.

[15]«Image Sensor Array: Fifth Embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of image sensor units arearrayed;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

light passing through the imaging lens and reaching each image sensorunit passes through the microlens and forms an image on the plurality ofimage sensors constituting the image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the image sensor unit.

[16]«Image Sensor Array: Sixth Embodiment»

An imaging apparatus including:

(A) an imaging lens; and

(B) an image sensor array in which a plurality of image sensor units arearrayed;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

light passing through the imaging lens and reaching each image sensorunit passes through the microlens and forms an image on the plurality ofimage sensors constituting the image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the image sensor unit.

[17]«Image Sensor Array: Fourth Embodiment»

An image sensor array including:

a plurality of image sensor units arrayed therein;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the image sensor unit.

[18]«Image Sensor Array: Fifth Embodiment»

An image sensor array including:

a plurality of image sensor units arrayed therein;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the image sensor unit.

[19]«Image Sensor Array: Sixth Embodiment»

An image sensor array including:

a plurality of image sensor units arrayed therein;

wherein a single image sensor unit includes a single microlens and aplurality of image sensors,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the image sensor unit.

[20]«Imaging Method: First Embodiment»

An imaging method using an imaging apparatus in which

(A) an imaging lens is provided, and

(B) an image sensor array in which a plurality of first image sensorunits and a plurality of second image sensor units are arrayed isprovided,

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

light passing through the imaging lens and reaching each first imagesensor unit passes through the first microlens and forms an image on theplurality of image sensors constituting the first image sensor unit,

light passing through the imaging lens and reaching each second imagesensor unit passes through the second microlens and forms an image onthe image sensor constituting the second image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the first image sensor unit, or

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the first image sensorunit, or alternatively,

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the first image sensor unit,

the first image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, the imaging method including:

generating electrical signals with the first image sensor and the thirdimage sensor in order to obtain one of either a right-eye image or aleft-eye image;

generating electrical signals with the second image sensor and thefourth image sensor in order to obtain the other of either a right-eyeimage or a left-eye image; and

outputting the electrical signals.

[21]«Imaging Method: Second Embodiment»

An imaging method using an imaging apparatus in which

(A) an imaging lens is provided, and

(B) an image sensor array in which a plurality of image sensor units arearrayed is provided,

wherein a single image sensor unit includes a single first microlens anda plurality of image sensors,

light passing through the imaging lens and reaching each image sensorunit passes through the microlens and forms an image on the plurality ofimage sensors constituting the image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the image sensor unit, or

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the image sensor unit, oralternatively,

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the image sensor unit,

the image sensor unit includes four image sensors, these being a firstimage sensor, a second image sensor, a third image sensor, and a fourthimage sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, the imaging method including:

generating electrical signals with the first image sensor and the thirdimage sensor in order to obtain one of either a right-eye image or aleft-eye image;

generating electrical signals with the second image sensor and thefourth image sensor in order to obtain the other of either a right-eyeimage or a left-eye image; and

outputting the electrical signals.

[22] The imaging method according to [20] or [21], wherein image datafor obtaining a right-eye image and image data for obtaining a left-eyeimage is obtained on the basis of a depth map created on the basis ofelectrical signals from the first image sensor and the third imagesensor as well as electrical signals from the second image sensor andthe fourth image sensor, and electrical signals output from imagesensors constituting second image sensor units.

[23]«Focusing Method: First Embodiment»

A focusing method using an imaging apparatus in which

(A) an imaging lens is provided, and

(B) an image sensor array in which a plurality of first image sensorunits and a plurality of second image sensor units are arrayed isprovided,

wherein a single first image sensor unit includes a single firstmicrolens and a plurality of image sensors,

a single second image sensor unit includes a single second microlens anda single image sensor,

light passing through the imaging lens and reaching each first imagesensor unit passes through the first microlens and forms an image on theplurality of image sensors constituting the first image sensor unit,

light passing through the imaging lens and reaching each second imagesensor unit passes through the second microlens and forms an image onthe image sensor constituting the second image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the first image sensor unit,

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the first image sensorunit, or alternatively,

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the first image sensor unit,

the first image sensor unit includes four image sensors, these being afirst image sensor, a second image sensor, a third image sensor, and afourth image sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, the focusing method including:

controlling the focusing of an imaging lens on the basis of image databased on electrical signals output from the first image sensor and thethird image sensor, and image data based on electrical signals outputfrom the second image sensor and the fourth image sensor.

[24]«Focusing Method: Second Embodiment»

A focusing method using an imaging apparatus in which

(A) an imaging lens is provided, and

(B) an image sensor array in which a plurality of image sensor units arearrayed is provided,

wherein a single image sensor unit includes a single first microlens anda plurality of image sensors,

light passing through the imaging lens and reaching each image sensorunit passes through the microlens and forms an image on the plurality ofimage sensors constituting the image sensor unit,

an inter-unit light shielding layer is formed between the image sensorunits themselves, and

a light shielding layer is not formed between the image sensor unitsthemselves which constitute the image sensor unit, or

an inter-device light shielding layer is formed only partially betweenthe image sensors themselves which constitute the image sensor unit, oralternatively,

an inter-device light shielding layer is formed between the image sensorunits themselves which constitute the image sensor unit,

the image sensor unit includes four image sensors, these being a firstimage sensor, a second image sensor, a third image sensor, and a fourthimage sensor, and

the first image sensor and the second image sensor are disposed along afirst direction, the third image sensor is adjacent to the first imagesensor along a second direction orthogonal to the first direction, andthe fourth image sensor is adjacent to the second image sensor along thesecond direction, the focusing method including:

controlling the focusing of an imaging lens on the basis of image databased on electrical signals output from the first image sensor and thethird image sensor, and image data based on electrical signals outputfrom the second image sensor and the fourth image sensor.

What is claimed is:
 1. A light detecting device, comprising: (a) a firstpixel unit comprising (i) a single and first microlens, (ii) a pluralityof pixels with a respective plurality of photoelectric conversionregions overlain by the first microlens, and (iii) a first light shield;and (b) a second pixel unit comprising (i) a single and second microlenswhich, in a plan view, is different in size than the first microlens,(ii) a single pixel with a respective single photoelectric conversionregion overlain by the second microlens, and (iii) a second lightshield, wherein, the first light shield includes a single and firstopening, the second light shield includes a single and second opening,the first microlens and the first opening correspond to the plurality ofphotoelectric conversion regions of the first pixel unit, the pluralityof photoelectric conversion regions of the first pixel unit and thesingle photoelectric conversion region of the second pixel unit aredisposed in a substrate, the first light shield is disposed between thesubstrate and the first microlens, the second light shield is disposedbetween the substrate and the second microlens, the plurality of pixelsof the first pixel unit comprises a first pixel and a second pixel, thefirst pixel and the second pixel of the first pixel unit and the singlepixel of the second pixel unit are disposed along a line extending alonga first direction and through a center of the single photoelectricconversion region of the second pixel unit, and the line extending alongthe first direction intersects the photoelectric conversion regions ofthe first pixel and the second pixel of the first pixel unit.
 2. A lightdetecting device, comprising: (a) a first pixel unit comprising (i) asingle and first microlens, (ii) a plurality of pixels with a respectiveplurality of photoelectric conversion regions overlain by the firstmicrolens, and (iii) a first light shield; and (b) a second pixel unitcomprising (i) a single and second microlens which, in a plan view, isdifferent in size than the first microlens, (ii) a single pixel with arespective single photoelectric conversion region overlain by the secondmicrolens, and (iii) a second light shield, wherein the plurality ofphotoelectric conversion regions of the first pixel unit comprises afirst photoelectric conversion region, a second photoelectric conversionregion, a third photoelectric conversion region, and a fourthphotoelectric conversion region, the third photoelectric conversionregion being disposed adjacent to the fourth photoelectric conversionregion along the first direction, the third photoelectric conversionregion being disposed adjacent to the first photoelectric conversionregion along a second direction, the fourth photoelectric conversionregion being disposed adjacent to the second photoelectric conversionregion along the second direction, and the second direction beingperpendicular to the first direction, the first light shield includes asingle and first opening, the second light shield includes a single andsecond opening, the first microlens and the first opening correspond tothe plurality of photoelectric conversion regions of the first pixelunit, the plurality of photoelectric conversion regions of the firstpixel unit and the single photoelectric conversion region of the secondpixel unit are disposed in a substrate, the first light shield isdisposed between the substrate and the first microlens, the second lightshield is disposed between the substrate and the second microlens, theplurality of pixels of the first pixel unit comprises a first pixel anda second pixel, and the first pixel, the second pixel, and the singlepixel of the second pixel unit are disposed in a line extending along afirst direction.
 3. The light detecting device of claim 1, wherein, inthe plan view, the first light shield is not between the plurality ofphotoelectric conversion regions of the first pixel unit.
 4. The lightdetecting device of claim 1, wherein, in the plan view, the firstmicrolens overlaps all of the plurality of photoelectric conversionregions of the first pixel unit.
 5. The light detecting device of claim1, wherein, in the plan view, the first opening overlaps the pluralityof photoelectric conversion regions of the first pixel unit.
 6. Thelight detecting device of claim 1, wherein, in the plan view, the firstmicrolens is larger than the second microlens.
 7. The light detectingdevice of claim 1, wherein, in the plan view, the first opening and thesecond opening have different sizes.
 8. The light detecting device ofclaim 7, wherein, in the plan view, the size of the first opening islarger than the size of the second opening.
 9. The light detectingdevice of claim 1, wherein the first pixel unit and the second pixelunit have footprints of different sizes.
 10. The light detecting deviceof claim 9, wherein the footprint of the first pixel unit is larger thanthe footprint of the second pixel unit.
 11. The light detecting deviceof claim 1, comprising a plurality of second pixel units, and a planararea of the first pixel unit is equal to a planar area of the pluralityof second pixel units.
 12. The light detecting device of claim 1,comprising a 1-pixel unit constituted by the second pixel units, whereinthe second pixel units are arranged in a Bayer array.
 13. The lightdetecting device of claim 1, wherein, in the plan view, one of theplurality of photoelectric conversion regions of the first pixel unitand the photoelectric conversion region of the second pixel unit areequal in area.
 14. The light detecting device of claim 1, wherein, inthe plan view, a width of the first light shield of the first pixel unitis larger than a width of the second light shield of the second pixelunit.
 15. The light detecting device of claim 1, wherein: a shortestdistance between one of the plurality of the photoelectric conversionregions of the first pixel unit and an outer periphery of the firstpixel unit is W1; a shortest distance between the photoelectricconversion region of the second pixel unit and an outer periphery of thesecond pixel unit is W2; and W1 is greater than W2.
 16. A lightdetecting device, comprising: (a) a first pixel unit comprising (i) asingle and first microlens, (ii) a plurality of photoelectric conversionregions, and (iii) a first light shield; and (b) a second pixel unitcomprising (i) a single and second microlens which, in a plan view, isdifferent in size than the first microlens, (ii) a single photoelectricconversion region, and (iii) a second light shield, wherein, the firstlight shield includes a single and first opening, the second lightshield includes a single and second opening, the first microlens and thefirst opening correspond to the plurality of photoelectric conversionregions of the first pixel unit, the plurality of photoelectricconversion regions of the first pixel unit and the single photoelectricconversion region of the second pixel unit are disposed in a substrate,the first light shield is disposed between the substrate and the firstmicrolens, the second light shield is disposed between the substrate andthe second microlens, and the first pixel unit is for parallax detectionand the second pixel unit is for image generation.
 17. The lightdetecting device of claim 1, wherein the first pixel unit furthercomprises a single and first color filter and the second pixel unitfurther comprises a single and second color filter.
 18. The lightdetecting device of claim 17, wherein, in the plan view, the first colorfilter is different in size than the second color filter.
 19. The lightdetecting device of claim 18, wherein, in the plan view, the first colorfilter is larger than the second color filter.
 20. The light detectingdevice of claim 17, wherein the first pixel unit is disposed adjacent tothe second pixel unit.
 21. The light detecting device of claim 20,wherein the first color filter is configured to transmit light of afirst color and the second color filter is configured to transmit lightof a second color different than the first color filter.
 22. The lightdetecting device of claim 21, wherein the first color is green.
 23. Thelight detecting device of claim 1, comprising a plurality of secondpixel units and the first pixel unit is interspersed among the pluralityof the second pixel units.
 24. The light detecting device of claim 1,wherein the first light shield and the second light shield are inelectrical communication.
 25. The light detecting device of claim 1,comprising a plurality of first pixel units at least some of which arearrayed along the first direction.
 26. The light detecting device ofclaim 25, wherein the first direction is a row direction.
 27. The lightdetecting device of claim 25, wherein the first direction is a columndirection.
 28. The light detecting device of claim 25, wherein at leastsome of the plurality of first pixel units are arrayed along a seconddirection, the second direction being perpendicular to the firstdirection.
 29. A light detecting apparatus comprising: an imaging lens;an image processor; and a light detecting device according to claim 1.30. The light detecting device of claim 20, wherein the first colorfilter and the second color filter are each configured to transmit lightof a first color.
 31. The light detecting device of claim 30, whereinthe first color is green.
 32. The light detecting device of claim 30,further comprising a third pixel unit comprising (i) a single and thirdmicrolens which, in a plan view, is different in size than the firstmicrolens, (ii) a single pixel with a respective single photoelectricconversion region overlain by the third microlens, and (iii) a thirdlight shield, the third pixel unit being disposed adjacent to the firstpixel unit, wherein, the single pixel of the third pixel unit also isdisposed along the line extending along the first direction, and theline extending along the first direction also extends through a centerof the single photoelectric conversion region of the third pixel unit.33. The light detecting device of claim 32, wherein the third pixel unitfurther comprises a single and third color filter, the third colorfilter being configured to transmit light of a second color differentthan the first color.
 34. The light detecting device of claim 33,wherein the first color is green and the second color is blue or red.